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ABSTRACT:  In  this  paper  vve  show  that  the  optical  flou\  a  2-D  field  that 
can  be  associated  with  the  variation  of  the  image  brigh  n*  ;s  pattern,  and  the 
2-D  motton  field ,  the  projection  on  the  image  plane  ot  the  3-D  velocity  field 
of  a  moving  scene,  are  in  general  different,  unless  very  special  conditions  are 
satisfied.  The  optical  flow,  therefore,  is  ill-suited  for  computing  structure 
from  motion  and  for  reconstructing  the  3-D  velocity  field,  problems  that 
require  an  accurate  estimate  of  the  2-D  motion  field.  We  then  suggest  a 
different  use  of  the  optical  flow.  We  argue  that  stable  qualitative  properties 
of  the  2-D  motion  field  give  useful  information  about  the  3-D  velocity  field 
and  the  3-D  structure  of  the  scene,  and  that  they  can  be  usually  obtained 
from  the  optical  flow.  To  support  this  approach  we  show  how  the  (smoothed) 
optical  flow  and  2-D  motion  field,  interpreted  as  vector  fields  tangent  to  flows 
of  planar  dynamical  systems,  may  have  the  same  qualitative  properties  from 
the  point  of  view  of  the  theory  of  structural  stability  of  dynamical  systems. 
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1.  Introduction 


A  key  task  for  many  vision  systems  is  to  extract  information  from  a  sequence 
of  images.  This  information  can  be  useful  to  solve  important  problems  such 
as  recovering  the  3-D  velocity  field,  or  segmenting  the  image  into  pans  cor¬ 
responding  to  different  moving  objects,  or  reconstructing  the  3-D  structure 
of  surfaces.  The  recovery  of  the  2-D  motion  field ,  (that  we  define  as  the 
projection  on  the  image  plane  of  the  3-D  velocity  field)  is  thought  to  be  an 
essential  step  in  the  solution  of  these  problems.  The  data  available,  how¬ 
ever,  are  temporal  variations  in  the  brightness  pattern.  These  variations  are 
usually  associated  with  a  perceived  motion  field,  called  optical  flow  (Gibson, 
1950;  Fennema  and  Thompson,  1979;  Horn  and  Schunck,  1981).  In  order 
to  recover  the  2-D  motion  field,  the  assumption  that  the  2-D  motion  field 
and  the  optical  flow  coincide  has  often  been  made.  It  must  be  noted  though, 
that  this  assumption  is  clearly  satisfied  only  in  the  case  in  which  variations 
in  the  brightness  pattern  correspond  to  features  of  the  visible,  3-D  surfaces. 
In  fact  several  authors  have  developed  algorithms  to  reconstruct  the  2-D  mo¬ 
tion  field  from  optical  flow  data  defined  only  at  locations  of  features  in  the 
image  (Hildreth,  1984a, b:  Waxman.  1986).  Examples  in  which  this  assump¬ 
tion  does  not  hold  are  known  (Horn,  1986).  but  they  have  been  regarded  as 
pathological  cases.  As  a  matter  of  fact,  algorithms  that  deal  with  the  recov¬ 
ery  of  the  2-D  motion  field  from  dense  optical  flow  data  have  been  proposed, 
with  the  more  or  less  explicit  assumption  that  the  two  fields  are  the  same 
(Horn  and  Schunck.  1981;  Nagel.  1984:  Kanatani,  1985). 

In  this  paper  W'e  show  that  the  optical  flow  and  the  motion  field  are  in 
general  different,  unless  very  special  conditions  are  satisfied.  We  explicitly 
compute  the  difference  between  their  normal  components  (the  component 
along  the  direction  of  the  gradient)  under  broad  assumptions.  We  show 
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that  they  are  arbitrarily  close  where  the  image  gradient  is  sufficiently  strong. 
Hence,  feature-based  matching  algorithms  that  rely  on  edges  of  various  types 
(including  texture  edges)  are  more  appropriate  than  point-to-point  ones  to 
solve  problems  that  rely  on  accurate  recovery  of  the  2-1)  motion  field,  such 
as  structure  from  motion.  One  may  then  ask.  what  is  the  optical  flow  for?  In 
the  second  part  of  the  paper  we  suggest  that  meaningful  information  about 
the  3-1)  velocity  field  and  the  3-D  structure  can  be  obtained  from  qualitative 
properties  of  the  2-f)  motion  field.  YVe  then  argue  that  this  information  ran 
be  retrieved  directly  from  the  optical  flow  or  its  normal  components.  \\e 
describe  a  specific  approach  that  exploits  results  from  the  theory  of  stability 
of  dynamical  systems.  A  more  detailed  analysis  of  this  approach  will  be 
presented  in  a  forthcoming  paper  by  V.  Torre  and  coworkers. 

The  paper  is  divided  in  two  parts.  In  the  first,  we  define  the  problem 
and  we  state  explicitly  the  assumptions  that  we  have  used.  In  particular,  we 
consider  in  detail  how  image  irradiance  can  be  related  to  scene  radiance  in 
the  case  of  a  scene  consisting  of  non-larnbertian  surfaces.  \Ve  describe,  then, 
a  method  that  allows  us  to  show  that  the  optical  flow  and  the  motion  field  are 
almost  always  different.  YVe  analytically  compute  the  difference  between  the 
normal  components  of  the  two  fields  assuming,  first,  the  lambertian  model 
of  reflectance  and  then  a  more  realistic  one  for  arbitrary  rigid  motion  of 
a  generic  surface.  YVe  also  calculate  how  this  difference  depends  on  the 
image  gradient  and  the  3-1)  velocity  of  moving  objects.  In  the  second  part 
we  show  how  both  the  optical  flow  and  the  motion  field  can  bo  processed 
to  become  vector  fields  tangent  to  flows  of  dynamical  systems.  The  optical 
flow  then,  can  be  considered  as  a  perturbed  motion  field  under  the  conditions 
determined  in  the  first  part.  Results  from  the  theory  of  stability  of  dynamical 
systems  suggest  that  qualitative,  stable  properties  <>l  the  motion  field  hold 
lor  the  optical  flow.  YVe  sketch  some  example  of  these  properties  and  how 
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they  can  be  used  in  a  description  of  the  3-D  velocity  field.  We  finally  discuss 
briefly  some  connections  with  biological  systems. 


2.  Preliminaries 


In  this  chapter  we  review  the  definitions  of  motion  field  and  optical  flow,  and 
we  state  the  assumptions  that  we  used  throughout  the  paper.  In  particular, 
we  consider  in  detail  how  image  irradiance  can  be  related  to  scene  radiance 
in  the  case  of  a  scene  consisting  of  non-lambertian  surfaces. 

2.1.  Definitions 


Let  us  define  notations  and  summarize  definitions  that  will  be  useful  in  what 
follows.  For  more  details  on  the  geometry  of  perspective  projection  see  Ap¬ 
pendix  A 1.  Throughout  the  following  we  will  assume,  if  it  is  not  otherwise 
stated,  that  anv  expression  can  be  differentiated  as  many  times  as  riedeed. 


Let 


xr  f-'  ^  n(x  (x  ■  n)n)  (2.1.1) 

be  the  equation  defining  the  projection  of  a  generic  point  on  the  image  plane, 
where  x?,  (x;,.y;,.0)  is  the  position  vector  of  the  projected  point,  x 

(x.y, z)  is  the  position  vector  of  the  point,  n  is  the  unit  vector  normal  to 
the  image  plane  (projection  plane)  and  f  is  the  focal  length  (see  Figure  1). 
Notice  that  the  origin  O  is  on  the  image  plane,  the  focus  of  projection  F  is 
located  at  (0.0.  /).  and  /n  *  x  is  the  vector  pointing  from  F  to  the  point. 


Figure  1.  The  geometry  of  perspective  projection. 


The  motion  field  vr  can  be  obtained  differentiating  (2.1.1)  with  respect 

to  the  time.  If  v  dxidt  we  have  1 

v,,  ,  ^  (v  (v-n)n  V  n  (x  (x-n)n))  (2.1.2) 

/  -  x  •  n  /  *  x  •  n 

Notice  that  in  (2.1.2)  v(,  is  given  in  terms  of  x  and  v.  position  and  velocity 
of  the  moving  points  in  the  scene,  which  are  not  known. 


Let  E  -  E(rl,,yl,.t)  be  the  image  irradiance,  that  is  the  intensity  of 
light  at  the  point  (xr .?/,,)  of  the  image  plan*  at  the  time  /.  If  V;,  is  the 


gradient  with  respect  to  the  image  coordinates,  then 

d  E  ()  E 
dl  dt 


VrE 


Now  if 


dE 

dt 


0 


(•2.1.3) 


(2.1,1) 


1  It  can  be  easily  shown  that  the  perspective  projection  of  the  3-D  velocity  vector 
is  equal  to  the  velocity  of  the  projected  point  on  the  image  plane,  since  both  the 
vector  are  defined  in  terms  of  infinitesimal.  This  is  not  true  for  a  generic.  Jiti it f 


vector 


t 

T 

L. 


then,  if  rVp£j;  =  0 

dEjm  _  VrE-vr 

;,V,,£ii  '"  VPE\,  1  •  • °> 

Therefore,  if  (2.1.4)  holds,  the  projection  of  the  motion  field  along  the  di¬ 


rection  of  the  gradient  can  be  given  in  terms  of  derivatives  of  the  image 
irradiance  (which  can  be  computed).  In  what  follows,  this  component  will 
be  called  vit  or  the  normal  component ;  thus 

v..-Vt,E  V.,E 

v.  e  v„f;  (2-'-6» 


Equation  (2.1.5)  can  be  interpretated  as  an  instance  of  the  well-known 
aperture  problem  (Marr  and  UUman,  1981;  Horn  and  Schunck,  1981):  that 
is  the  information  available  at  each  point  of  a  sequence  of  frames  is  only 
the  component  of  the  motion  field  along  the  direction  of  the  image  gradient. 
To  recover  a  full  and  unique  motion  field,  some  other  constraint  is  needed: 
Horn  and  Schunck  (1981).  for  example,  showed  that  there  is  only  one  2- 
D  field  whose  normal  component  coincides  with  (2.1.6)  and  which  is  the 
smoothest  of  all  possible  ones.  Examples  for  which  (2.1.5)  is  not  true  are 
well  known  (Horn  and  Schunck.  1981).  Consider,  for  instance,  a  rotating 
sphere  with  no  texture  on  it  (i.e.  with  uniform  albedo)  under  arbitrary, 
fixed  illumination.  Since  the  image  irradiance  at  each  image  location  does 
not  change  with  time,  the  left-hand  side  of  (2. 1 .5)  is  identically  equal  to  zero, 
while  the  right-hand  side  is  different  fr  mi  zero  almost  everywhere.  Notice 
that  keeping  the  sphere  fixed  and  moving  the  light  source  (2.1.5)  is  again 
wrong.  In  this  case,  however,  the  left-hand-side  is  different  from  zero  while 
v  is  zero  everywhere.  In  both  cases  the  perceived  motion  in  the  image  is 
different  from  the  motion  field.  It  is  worthwhile,  then,  to  introduce  a  new 
field,  called  the  minimal  optical  flow,  related  to  the  perceived  motion  in  the 
image,  and  not  necessarily  equal  to  t  he  normal  component  of  t  he  mot  ion  field . 
Notice  that  the  perceived  motion  in  the  examples  above  agrees  qualitatively 
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with  the  left-hand-side  of  (2.1.5).  Indeed,  the  optical  flow  in  the  first  case 
is  identically  equal  to  zero,  while  in  the  second  is  different  from  zero  almost 
everywhere.  Therefore  let  us  define  the  normal  component  Of  of  the  optical 
flow  as:  * 


Of 


dEdt  V„E 


(2.1.7) 


\V,,E\:  V,E 

Hence,  with  respect  to  this  definition,  the  minimal  optical  flow  and  the  nor¬ 
mal  component  of  the  motion  field  are  always  directed  along  the  gradient 
and  they  coincide  if  and  only  if  (2.1.4)  holds. 


Remark:  in  the  literature,  it  is  usually  assumed  that  (2.1.4)  holds.  As  a 
consequence,  the  normal  components  of  the  motion  field  and  of  the  optical 
flow  are  the  same  and  the  latter  can  be  used  as  a  constraint  to  recover  the 
2-1)  motion  field. 


2.2.  Scone  Radiance  and  Image  Irradiance 

Let  us  review  briefly  some  definitions  of  photometry  and  make  explicit  the 
constraints  under  which  the  image  irradiance  is  related  to  the  scene  radiance. 
The  image  irradiance  E  is  the  power  per  unit  area  of  light  at  each  point 
(sr.yr)  of  the  image  plane:  thus  E  E(rl..yv).  The  scene  radiance  L 
is  the  power  per  unit  area  of  light  that  can  he  thought  emitted  by  each 
point  of  a  surface  .S'  in  the  scene  in  a  particular  direction.  This  surface 
can  be  fictitious,  or  it  may  be  the  actual  radiating  surface  of  a  light  source, 
or  the  illuminated  surface  of  a  solid.  The  scene  radiance  can  be  thought 
as  a  function  of  tfie  point  of  the  surface  and  of  the  direction  in  space.  If 
(a.b)  are  intrinsic  coordinates  of  the  surface  and  (o.,f)  polar  coordinates 
determining  a  direction  in  space  with  respect  to  the  normal  to  the  surface. 


we  can  write  /,  L{a.b.  n..f).  (liven  the  seem1  radiance  it  is  possible,  in 
principle,  to  compute  the  expected  image  irradiance.  lor  example  in  the 


Figure  2  Scene  radiance  and  image  irradiance  in  the  pinhole  approximation:  the 
image  irradiance  at  the  point  is  given  by  the  scene  radiance  at  the  point 

(a,  6)  on  the  surface  in  the  direction  of  the  line  connecting  the  two  points  and 
passing  through  the  pinhole  P ^ 


case  of  pinhole  camera  approximation,  that  is  assuming  that  the  camera  has 
an  infinitesimally  small  aperture,  the  image  irradiance  at  a  point  (xr.yr)  is 
proportional  to  the  scene  radiance  at  the  point  (n.b)  on  the  surface  in  the 
direction  of  the  pinhole,  say  (o where  (x,  .  </,  ).  (n.b)  and  the  pinhole 
lie  on  the  same  line  (see  Figure  2).  Therefore  we  ha\e 
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where  (j,,(o.h),j/,.(o.fc))  is  the  image  point  that  lies  on  the  line  connecting 
(a,  6)  to  the  pinhole.  In  practice,  however,  the  aperture  of  any  real  optical 
device  is  finite  and  not  very  small  (ultimately  to  avoid  diffraction  effects): 
thus  (2.2.1)  does  riot  generally  hold.  Assuming  that  the  surface  is  lambertian, 
i.e.  L(a.b.a.J)  L(a,b).  that  there  are  not  losses  within  the  system  and 
that  the  angular  aperture  (on  the  image  side)  is  small  it  can  be  proved  (Born 
and  Wolf.  1 959)  that 


E[Tj  (a.  6).  yy  (a.  b))  --  L(a.  6)0  cos1 


(2.2.2) 


where  0  is  the  solid  angle  corresponding  to  the  angular  aperture  and  p  is 
the  angle  between  the  principal  ray  (that  is  the  ray  passing  through  the 
renter  of  the  aperture)  and  the  optical  axis.  With  the  further  assumption 
that  the  aperture  is  much  smaller  than  the  distance  of  the  viewed  surface, 
the  lambertian  hypothesis  can  be  relaxed  to  give  (Horn  and  Sjoberg.  1979) 

E(xr(a,  b).  yr  (a,  b))  -  L(a.  6,  o .i'  )0  cos  1  p  (2.2.2) 


where  o  '  and  d"  are  the  polar  coordinates  of  the  direction  of  the  principal 
ray.  It  must  be  pointed  out  that  (2.2.2)  holds  if  L  is  continuous  with  respect 
to  o  and  d.  In  what  follows  we  will  assume  that  this  is  the  case.  Furthermore, 
we  will  assume  that  the  optical  system  has  been  calibrated  so  that  (2.2.2) 
can  be  rewritten  as  (2.2.1).  Finally,  notice  that 


Vs/, 


da  db 
dt  '  dt  1 


where  V.s  is  the  gradient  with  respect  to  the  surface  coordinates,  since  dif¬ 
ferentiating  (2.2.1)  we  have 

V  •  (d.r.  .dt/))  V.s  /,  ■  (da.  db) .  (2.2.51 


L  1  <  I',  4'-  4 k.  4  . 


■  4*4  «  ri.l't.t’tJ'M  l.l  l.i'l.l'ft.t'l.VlJ  t.t  h*J»*  li*  *,• '*••‘4.* I 


3.  Minimal  optical  flow  and  Motion  Field 


We  describe  a  general  method  that  allows  us  to  show  that  the  minimal  optical 
flow  and  the  normal  component  of  the  motion  field  are  almost  always  differ¬ 
ent,  or  equivalently  that  (2.1.4)  does  not  hold.  We  compute  the  difference 
between  the  normal  components  of  the  two  fields,  assuming  first  the  Lamber¬ 
tian  model  of  reflectance  and  then  a  more  realistic  one  for  pure  translation, 
pure  rotation  and  general  rigid  motion  of  a  generic  surface.  It  turns  out 
that  the  two  fields  are  equal  only  under  very  special  conditions,  which  can 
be  explicitly  stated.  We  also  show  that  the  difference  is  smaller  where  the 
image  gradient  is  stronger,  justifying  the  use  of  feature-based  algorithms.  Of 
course,  this  argument  does  not  imply  that  feature-based  algorithms  should  be 
used:  it  says,  however,  that  locations  of  edges  (meant  here  as  sharp  changes 
in  intensity)  contain  most  of  the  correct  information. 


3.1.  Computing  the  Minimal  Optical  Flow 

Consider  a  rigid  surface  .S'  moving  in  space  from  (2.2.1 ).  The  image  irradiance 
K  at  the  time  I  at  the  point  (j is  equal  to  the  scene  radiance  L  at  the 
point  (a.  6)  on  S.  i.e.  L(<i-b).  The  image  irradiance  at  the 

time  /  •  A/  is  given  by  the  scene  radiance  of  the  surface  at  the  time  I  •  At. 
As  shown  in  f  igure  3.  the  point1  on  .S'  that  radiates  toward  (j ',..yr)  at  the 
time  t  ■  At  is  the  point  (a  So  b  Ah).l  The  normal  N  to  .S’  at  the  time 
/  *  A/  at  the  point  (a  Ah).  N >  .  .\t(u  Ao.h  Ah).  will  he 


N<.^/(«  A«.h  Ah)  N,(«  At/.h  Ah)  ■  AN  (3.1.1) 

I  We  assume  that  I  tie  surface  corresponds  to  a  moving  convex  body  to  avoid  self- 
occlusions  due  to  the  motion  In  fact  the  computation  that  follows  holds  for 
any  convex  surface  patch 


wvw 


Figure  ?>  Computing  the  minimal  optical  flow:  the  point  ( a.b )  on  S  radiates  toward 
)  at  time  /  The  point  (a  Aa.b  Ab)  radiates  toward  the  same  point  at 
time  I  ■  At.  The  normal  N|  is  the  normal  to  the  5  at  the  point  and  N-j  at 

( a  Aa.b  Ab) 

where  AN  is  the  first  order  variation  of  N  due  to  the  motion  of  .S'  during 
the  time  interval  A/.  Now  in  t  lie  case  of  translation 

AN  0  (o.l.2) 

while  in  case  of  rotation  with  angular  velocity  u> 

AN  u  ■  N At  (TIAI 

Notice  that  can  he  considered  as  the  expression  ol  AN  for  any 


kind  of  motion.  Similarly,  for  each  argument  A  of  the  scene  radiance,  we  can 
write 


-  At{a,b)  -4  AA.  (3.1.4) 

To  compute  A  A,  let  us  distinguish  between  arguments  of  L  that  are 
intrinsic  function  of  the  surface  coordinates  (a,  6),  such  as  texture  and  albedo, 
and  those  that  are  in  fact  function  of  the  space  coordinates  (r.y,c),  (such 
as  the  illumination  and  the  point  of  view)  and  that  are  expressed  in  terms 
of  ( a,b )  only  for  convenience.  If  A  is  an  intrinsic  function  of  the  surface 
coordinates,  it  follows  immediately  that 

AA  =  0,  (3.1.5) 

while  if  A  is  a  function  of  the  space  coordinates,  from  the  Taylor  expansion 
we  have 

AA^VAvAl.  (3.1.0) 

where  V  is  the  gradient  operator  with  respect  to  the  space  coordinates.  Let 

us  assume  that  L  can  be  written  as  a  funct  ion  of  m  arguments  A' .  i  1 . in 

and  of  N.  Then,  taking  into  account  (3.1.3)  and  (3.1.4).  (‘2.2.1)  becomes 
E(x},,yfl,t  -  At)  L(A\(a  -  Aa,b  Ab)  t  A.T.N,(o  A a.b  Ab)  •  AN) 

(3.1.7) 

at  time  t  -  At  and 

E(x,„y,,J)  L(A\(a,  6).  A(  (a.b))  (3.1.8) 

at  time  t.  Therefore,  using  (3.1.6)  and  (3.1.7), 

dE 

dt 

Jim  ^  (L(A){a  Aa.b  Ab)  t  A.4'.Nf(«  Aa.b  Ab)  AN) 

L(A)(a.b).N,(a.b))),  (3.1.9) 

where  the  A/t!  are  computed  using  (3.1.5)  or  (3.1.0)  according  to  the  kind 

of  argument.  From  (3.1.9).  the  minimal  optical  flow  can  be  derived  easil\. 
To  simplify  notation,  let  us  suppress  the  subscript  I  from  liquation  (3.1.9). 
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Remark:  in  the  case  of  non-uniform  illumination  the  right  hand  side  of  (3.2.2) 
contains  an  extra  term  due  to  AI.  Using  (3.1.6)  to  compute  the  components 
of  AI,  (3.1.10)  yields 

1  /  d\  dx  d\  dy  d\  dz  \ 


v±  -  0F  - 
which  can  be  rewritten 


\dx  dt  dy  dt  dz  dt. 


5  XT 

V],E  Pdt  ' 


)‘N’ 


(3.2.3) 


since  dl/dt  -  0  (the  illumination  is  supposed  to  be  fixed).  Let  us  consider 
now  the  case  of  a  rotating  lambertian  surface. 

3.3.  Rotation  of  a  Lambertian  Surface 

Let  S  be  a  lambertian  surface  rotating  in  space  with  angular  velocity  u>.  Let 
I  be  again  uniform.  Applying  the  same  argument  of  the  previous  section  but 
taking  into  account  the  constraint  (3.1.3)  for  V.V,  we  get 


r .  (h 


/>N  •  I  ■  u 

?,.£!  ' ' 


(3.3.1) 


In  the  case  of  rotation,  therefore,  even  under  uniform  illumination,  the 
minimal  optical  flow  and  the  normal  component  of  the  motion  field  are  dif¬ 
ferent.  They  are  equal  for  any  surface  only  if  uj  and  I  are  parallel.  This 
corresponds  to  the  case  of  a  surface  rotating  around  an  axis  parallel  to  the 
direction  of  uniform  illumination.  In  the  rase  of  non-uniform  illumination, 
an  extra  term  like  the  one  in  (3.2.3)  must  be  added  to  (3.3.1).  Remark:  it 
is  worth  considering  analytically  the  example  of  the  rolating  sphere  of  the 
previous  section.  Due  to  rotational  symrnetn  we  have 

N(o  An, 6  Ah)  u  ■  N (a  Aa.b  Ah)A/  N(a.h)  (3.3.2) 
va.Vfeon  the  sphere.  Furthermore. 

Ii.^f(fl  A  a.b  Ah)  !,(«  A  a.b  Ah)  AI  l,(a.b).  (3.3.3) 


since  in  this  case  the  displacement  in  space.  vA t,  is  equal  to  the  displacement 


on  the  surface,  (Aa,A6).  Therefore,  if  p  is  uniform. 


(3.3.4) 


The  minimal  optical  flow  is  then,  as  expected,  equal  to  zero  under  any  illu¬ 


mination. 


3.4.  Translation  of  a  Specular  Surface 


Let  us  consider  now  a  model  of  reflectance  more  realistic  t  han  the  lambertian 
one.  Following  Phong  (1975;  see  also  Horn  and  Sjoberg,  1979)  we  define  the 
scene  radiance  as  a  linear  combination  of  a  lambertian  and  a  specular  term. 


L  L\mnh  '  ^spec- 


(3,1.1) 


The  lambertian  term  is  equal  to  the  one  used  before,  while  the  specular  term 


D  R 


{:»•  i  •-) 


where  s  is  the  fraction  of  light  reflected  by  the  surface.  D  / n  x  is  the 

vector  pointing  from  the  focus  to  the  radiating  point  and 

R  I  2(1  -N)N  (3.1.3) 

is  the  unit  vector  in  the  direction  of  the  perfect  specular  reflection.  Let  us 
assume  that  s  is  not  a  function  of  the  direction  of  the  incident  light  and  that 
it  is  constant  on  the  surface.  The  specular  term  is  thus  proportional  to  the 
cosine  of  the  angle  between  the  direction  of  specular  reflection  and  the  line 
of  sight . 

Since  we  are  computing  derivatives  and  L  is  a  linear  combination  of 
Li,nni  and  we  can  compute  separately  the  contributions  to  the  min¬ 

imal  optical  flow  due  to  the  lambertian  and  the  specular  term,  adding  the 


1G 

results  afterward.  Therefore,  we  only  need  to  compute  now  the  specular  one. 
Let  us  consider,  first,  the  case  of  pure  translation  of  a  surface  S  radiating 
accordingly  to  (3.4.2)  and  let  us  call  5  a  specular  surface.  If  S  is  translating 
with  velocity  v  and  I  is  uniform,  substituting  (3.4.2)  into  (3.1.10)  and  taking 
into  account  the  constraint  (3.1.2),  we  have 


0/r 

since  from  (3.1.6) 


s_  (D2\  ■  R  -  (D  •  v)(D  •  R)) 
Da  ||Vr£j 


(3.4.4) 


AD  dT)  dx  <9D  dy  dD  dz  dT)  dx 

At  — A t  dx  dt  dy  dt  ^  dz  dt  dt  dt 

Using  again  the  two  fields  we  get  a  well  known  vector  identity: 


O, 


(3.4.6) 


.s  (v  >  D)  •  (R  ■  D) 

Da  V,.E 

Thus,  in  the  case  of  translation  of  a  specular  surface,  the  minimal  optical 
flow  and  the  normal  component  of  the  motion  field  are  always  different. 


Remark:  let  us  consider  the  case  of  orthographic  projection.  When  f  ■-*  oc, 
(3.4.6)  becomes 

f,i  Of  -  0. 

since  when  /  -■*  oc,  I)  -  oc.  Therefore,  in  the  othographic  limit,  the 
minimal  optical  flow  of  a  translating  specular  surface  is  equal  to  the  normal 
component  of  the  2-D  motion  field. 


3.5.  Rotation  of  a  Specular  Surface 

Consider  now  the  same  specular  surface  S  rotating  in  space  with  angular 
velocity  u>.  Then,  substituting  (3.4.2)  into  (3.1.10)  and  taking  into  account 
the  constraint  (3.1.3).  we  have 


V 


-w 


*.t .  ir.  v  *t  '‘*r,  •  .1 


v,  0F  =.  —  - „  (2D2 ({1  ■  N)(D  •  u  -  N)  -  (D  •  N)(I  oj  ■  N)) 
V(,E  D’' 

((«  ■  x)  ■  D)  •  (R  -  D)  - 

((«  >  X„)  y  D)  (R  >  D)),  (3.5.1) 

since  v  u>  •  x  and  Xn  gives  the  location  of  the  axis  of  rotation.  Now 
(3.5.1)  gives 

v_  0F  ^  £S  Dj  (2D^(I  •  N)(D  •  u  x  N)  ^  2D2(D  •  N)(I  •  w  x  N) - 
~D2({ w  x  x)  ■  R)  -  (D-R)(D-w  x  x)  -  2D2({u>  ,  x„)  >  D)  ■  (R  x  D)) 
This  expression  can  be  simplified  in  the  following  way.  since  D  =  / n-^x, 


v  Of  "S  3  (‘2/D2(I  -N)(n  u>  *  N)  ■  20‘(I-N)(x-w  -  N) 

V ;,E  D" 

*  2  //*  ( D  •  N )  ( I  •  w )  •  2/>'((w  •  x„)  •  D) 

(R  •  D)). 

that  can  be  rearranged  to  give 


v  Of 


V,  E  D 


3  ( /( n  •  w)  •  (2/2* (I  •  N)N  •  (D  •  R)x)  • 


D2( I  •  w)  •  (2(D-N)N  x)  •  20:’((w  ■  x  .)  ■  D I  (R  ■  D)). 


f)iit  x  fn  D:  t  herefore. 


v  Of 


V,  E  D 


,  (n  -  u)  ■  (2f  D‘  (1  •  N)N  •  /(D  •  R)D) 


•(I  ■  u)  ■  (2D'(D  ■  N)N  DJD  ■  f  I)2  n)  ■  2  //*((w  •  x  )  ■  D)  ■  (R  ■  D) ) . 


That  is. 


>.»*«  .■  I  .'I 


Vi  -  Of  -  —^5  ((n  >  w)  •  (2/D2  (I  •  N)N  *  /( D  •  R)D)-r 
(Ixw)-(-  D2(D-2(D-N)N)  •  /D2n)  ~  2D2 ((v  ■  x,)vD)-(R  -  D)) 


Since 


(I  *  <j)  •  N  (N  *  ijj)  ■  I, 


we  have 


Of  =--  _r 3  ((n  >  w)  ■  (2/D* (I  •  N)N  ■  /(D  •  R)D  -  /D2I) 

-(I  X  w)  •  (  -  D2(D  2(D  •  N)N)) J- 
42D2((w  -  x„)  ■  D)  (R  •  D)). 

but  I  -  2(1 -N)N  R,  and  so 


l’-  °F  IP  V,D  '  W)-|D  -MD-NJN) 

f{n  ■  w)  -  (D  ■  (D  ■  R) )  •  2D2((u/  •  x,  )  •  D)  -(R.  •  D)).  (3.5.2) 

The  minimal  optical  flow,  therefore,  is  equal  to  the  motion  Held  for  any 
specular  surface  only  when  I.  u  and  u  are  parallel. 

Remark:  let  us  consider.  aga:n.  the  orthographic  limit,  'faking  into  account 
that  as  /  •  x.  1)  x  and  D  / )  n.  (3.5.2)  becomes 

»•  <>f-  ((n-N)lu;  I-N)  •  (I-N)(w  ■  ii  -  N ) )  (3.5.3) 

\  ,  h 

Therefore,  even  under  othographic  limit,  the  two  fields  are  different. 


3.6.  General  Gase 


bet  us  consider,  now  .  t  lie  general  caM\  We  w  ill  assume  (3.1.1)  as  scene 
radiance  of  a  surface  >  undergoing  a  given  rigid  motion  (composition  of  a 
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rotation  and  a  translation)  in  space.  Adding  together  (3.3.1).  (3.4.6)  and 
(3.5.2).  v\t  obtain  the  difference  between  the  motion  field  and  the  minimal 
optical  flow  for  a  surface  in  the  general  case  under  uniform  illumination,  i . e . : 

pN  •  I  *  w  *  (v  •  D)  •  (R  ■  D) 

r-  °F  v,  /•;  '  i):’  v,./-; 

•  iy,  *  (/>'(  1  ■  u;)-(D  2(D  •  N)N) 

/( n  •  w)  •  (D  >  (D  ■  R.) )  •  2l)~  ( (<j  ■  x  ,)  '  D)  •  (R.  ■  D)).  (3.6.1) 

The  right-hand  side  of  (3.6.1)  is  generally  different  from  zero.  In  fact, 
there  are  no  general  conditions  under  which  it  is  identically  equal  to  zero. 
Notice,  however,  that  if  *.•  and  r  are  bounded 

lim  r  ()F  0.  (3.6.2) 

V,K  -oc 

Kquation  3.6.2  shows  that  the  points  in  the  image  where  the  gradient  is 
stronger  are  the  points  when'  the  minimal  opt ical  flow  is  closer  to  the  motion 
field.  These  points  are  characterized  by  sharp  changes  in  intensity  -  edges 
-.  that  usually  correspond  to  important  physical  events  on  surfaces,  such 
as  boundaries,  orientation  discontinuities  and  especially  surface  markings. 
Thus,  to  solve  problems  such  as  structure  from  motion,  or  the  recovery  of 
the  3-1)  velocity  held,  which  require  an  accurate  estimate  of  the  2-1)  motion 
field,  edge-based  algorithms  seem  more  suitable  than  algorithms  based  on 
spatial  and  temporal  derivatives  of  t  ho  image  brightness.  As  a  consequence, 
in  order  to  obt  ain  a  precise  reconst  met  ion  of  1  he  2-1)  mot  ion  field,  algorit  hms 
based  on  the  solution  of  the  correspondence  problem  among  edges  ma\  be 
u-ed.  Notice  that  mutrhmy  (  an  be  host  performed  between  frames  lliat  are 
i  losoK  spaced  in  time  whereas  the  structure  from  motion  computation  is 
l>est  performed  between  w  idol\  spaced  frames.  The  whole  argument  ago  r- 
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with  the  fact  that,  as  intuitively  expected,  the  minimal  optical  flow  and  the 
motion  field  at  image  features  corresponding  to  precise  locations  on  the  3-D 
surfaces  coincide.  It  must  be  pointed  out  that  in  this  analysis  we  have  not 
considered  shadows  and  self-shadow  effects.  They  usually  give  rise  to  edges 
in  the  image  that  do  not  correspond  to  features  in  the  scene.  Furthermore, 
the  Phong  model  of  reflectance  does  not  include  sharp  intensity  changes  due 
to  specularities. 


V,' 


4.  Qualitative  Properties  of  the  Minimal  optical  flow 


Traditionally,  the  optical  flow  has  been  considered  as  the  first  step  for  recov¬ 
ering  3-D  structure  and  3-D  motion.  In  this  chapter  we  suggest  a  different 
use  of  the  minimal  optical  flow.  We  argue  that  qualitative  properties  of  the 
2-D  motion  field  give  useful  information  about  the  3-D  velocity  and  the  3-1) 
structure  of  surfaces  and  that  these  qualitative  properties  can  be  usefully 
inferred  from  the  obtainable  minimal  optical  flow.  As  an  example  of  this 
approach,  we  introduce  the  qualitative  properties  associated  with  2-D  dy¬ 
namical  systems  and  show  how  to  process  minimal  optical  flow  and  motion 
field  for  making  them  equivalent  to  flows  of  dynamical  systems  on  the  plane. 
We  then  suggest,  from  properties  of  structural  stability  of  dynamical  sys¬ 
tems.  that  the  minimal  optical  flow  may  be  equivalent  to  the  motion  field  in 
terms  of  qualitative  properties. 

4.1.  What  is  the  minimal  optical  flow  for? 

In  the  previous  section  we  have  shown  that  the  minimal  optical  flow  and 
the  motion  field  are  different  almost  everywhere.  As  a  consequence,  the 
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minimal  optical  flow  cannot  bo  used  to  solve  problem'-  Midi  as  structure 
from  motion  and  recovery  of  the  3-1)  velocity  field,  w  hose  solutions  rely  on 
precise  reconstruction  of  the  2-1)  motion  field.  We  have  also  proved  thal  the 
two  fields  are  very  similar  at  locations  where  the  image  gradient  is  strong. 
This  led  to  the  suggestion  that  feature-based  algorithms  may  provide  more 
reliable  solutions  to  those  problems. 

Here  we  argue  that  the  minimal  optical  flow,  as  a  field  defined  almost 
everywhere,  can  be  used  to  retrieve  meaningful  information  about  the  3-1) 
velocity  field  and  the  3-1)  structure  of  the  scene.  In  particular,  we  consider 
qualitative  properties  of  the  2-D  motion  fields  which  can  be  connected  to 
significative  events  in  the  scene.  Such  properties  are  likely  to  be  found  in 
the  corresponding  minimal  optical  flows  as  well.  As  an  example,  consider  an 
object  moving  toward  the  image  plane.  This  kind  of  motion  generates  a  focus 
of  expansion  in  the  2-1)  motion  field.  The  presence  of  a  focus  of  expansion 
on  the  image  plane,  therefore,  may  be  related  to  an  object  moving  toward 
the  plane  itself.  As  we  have  seen,  however,  the  information  available  is  not 
the  motion  field,  nor  its  normal  component,  but  the  minimal  optical  flow  (or 
its  nomal  component).  If  the  difference  between  the  two  fields  is  sufficiently 
small,  we  ex  peel  to  find  a  focus  of  expansion  also  in  t  he  minimal  opt  ic.tl  How  . 
In  the  next  sect  ions  we  will  show  how  I  he  2-1)  motion  field  and  t  he  opt  i<  a.  I  flow 
can  be  considered  vector  fields  tangent  to  flows  of  some  d\ mimical  system:  it 
becomes  then  possible  to  establish  a  suggestive  analog)  between  the  theorv 
ol  structural  stability  ol  dynamical  systems  and  the  qualitative  description 
of  the  two  fields.  A  focus  of  expansion  of  a  dynamical  system.  for  example, 
i-  a  stable  property  for  small  perturbations  of  the  system:  this  means  that 
gi'en  a  vector  field  with  a  focus  of  expansion,  every  field  obtained  from  it  by 
means  <>l  a  sufficiently  small  pert  lir  bat  ion  will  also  show  a  locus  of «  x  pa  to  ion . 


I  A  .  «  ■  ►  A  a4 


.  *  |.«  *.!■*.»  ».»  !.<  l.l  K<  |a*  l.« 


1 1<  *ii'J  it’ 


4.2.  Smoothing  the  Optical  Flow  and  the  Motion  Field 

In  order  to  establish  a  connection  with  the  theory  of  stability  of  dynamical 
systems,  we  must  insure  that  the  optical  flow  and  the  motion  field  have  an 
appropriate  degree  of  smoothness.  This  is  not  always  the  case,  because  of 
discontinuities  arising  at  object  boundaries  or  to  noise  affecting  the  optical 
flow  data.  We  suggest  to  use  a  filtering  step  to  smooth  the  field.  It  is 
worthwhile  noticing  that  a  filtering  step  on  the  normal  component  of  a  dense 
motion  field  is  a  (regularization)  method  to  recover  the  whole  2-D  motion 
field.  J 

4.3.  Qualitative  Descriptions  of  Dynamical  Systems 


For  a  rigorous  and  thorough  review  on  dynamical  systems  see  Hirsch  and 
Smale  (1974).  Here,  for  the  sake  of  completeness,  we  summarize  the  main 
definitions  and  results. 

A  dynamical  system  is  a  (' 1  map  O:  H  ■  .1  •  .4.  where  .1  is  an  open 

set  of  an  Fuclideari  space  and  writing  o[t.r)  Ot(r).  the  map  Otm.  .1  -  .4 

sat  islies: 

(a)  o i  :  .4  •  .4  is  the  identity: 

(b)  t he  composit ion  o,(o..(x))  Or  . ...  for  each  t.s  ■  R. 

A  dynamical  system  Ot  on  -4  gives  rise  to  a  differentia)  equation  on  .4. 
that  is  a  vector  field  y:  A  •  K  defined  as  follows: 

y(-r)  j.oi(/)  (i.:i.i) 

dt  , 

^This  "smoothed"  2-1)  motion  field  may  not  be  the  same  recovered  using  standard 
algorithms,  but  its  qualitative  properties  are  likely  to  be  preserved.  The  analogy 
we  are  about  to  present,  indeed,  will  support  this  argument  (and  the  equivalence 
between  qualitative  properties  of  the  2-P  motion  field  and  t  fie  optical  flow  a*- 

well ) 


Thus,  for  every  x.  t/(x)  is  the  tangent  vector  to  t  lie  curve  t  ■  ot(x)  at  /  0. 

Equation  (4.3.1)  can  be  rewritten  in  a  more  conventional  way  as 

dx 

di  -  y(r).  (f.3.2) 

Inder  suitable  conditions  on  y(x).  tliere  exists  a  dynamical  system  associ¬ 
ated  to  (4.3.2)  as  a  differential  equation.  Namely,  a  sufficient  condition  on 
y(x)  is  that  it  is  a  C1  function  defined  on  an  open  subset  of  R~ .  Intuitively  a 
dynamical  system  can  be  thought  as  a  one-parameter  family  of  transforma¬ 
tion  Qt :  A  ♦  A  describing  the  motion  of  the  points  in  .4  as  the  time  passes. 
The  trajectories  of  the  points  are  given  by  the  solution  curves  to  equation 
(4.3.2).  Since  equation  (4.3.2)  is  autonomous  (that  is,  the  right-hand  side 
does  not  depend  explicitly  on  time),  if  y(x")  0.  then  x  x1'  is  a  solution 

to  it.  Without  loss  of  generality,  we  can  assume  that  x"  coincides  with  the 
origin.  For  obvious  reasons,  we  will  restrict  our  attention  to  planar  systems. 
(i.e.  in  what  follows.  .4  will  be  art  open  set  in  }{~).  Solutions  like  x‘  art' 
called  equilibrium  points  or  equilibria.  In  the  case  of  linear  s\ stems,  useful 
qualitative  information  about  the  behaviour  of  the  solution  to  (4.2.2)  can  be 
obtained  from  the  eigenvalues  of  the  matrix  M  of  the  coefficients  of  the  differ¬ 
ential  equation.  The  restriction  to  planar  systems  reduces  the  classification 
to  four  fundamental  cases: 

/  :  Af  has  real  eigenvalues  of  opposite  signs.  In  this  rase  the  origin  is  called 
a  saddle:  the  equilibrium  i1-  unstable  (an  equilibrium  is  stable  if  arn 
nearby  solutions  to  it  sta\s  nearb\  for  all  the  future  time.  It  i-  unstable 
ol  lierwise). 

/  /  :  The  eigenvalues  have  negative  real  parts.  The  origin  is  called  a  ami-  and 
it  is  stable  equilibrium.  The  main  property  of  a  sink  is  that 


/ 

t  I 

Figure  4.  \ector  field  tangent  to  a  planar  sink:  all  the  solutions  curves  are  pointing 
toward  the  origin. 


« 

« 
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Qualitatively,  the  phase  jiorlrait  of  the  solutions,  that  is.  the  family  of  the 
solutions  curves  as  a  subset  of  ft'.  looks  like  Figure  ).  where  only  some 
tangent  vectors  of  some  solutions  curves  have  been  drawn.  Sinks  can  be 
classified  depending  on  further  charade,  sties  of  the  eigenvalues.  A  focus 
(Figure  I).  for  example,  represents  the  case  of  coincident  eigenvalues 
(A/  is  supposed  to  be  diagonali/.able):  a  node,  the  case  of  different  rial 
eigenvalue;  a  spiral,  the  case  of  complex  conjugates  eigenvalues.  \  sink- 
increasing  rotational  component  corresponds  to  each  different  case. 

Ill  The  eigenvalues  have  positive  real  parts.  The  origin  i"  called  a  sourct . 
The  main  proper!  \  of  a  source  is  that 

lint  x{t)  x 
r  ■  \ 

and 

(  lint  s(t)  0. 

A  source  can  be  considered  as  t  he  dual  case  of  a  sink:  t  lie  phase  port  rail 
of  a  source  and  of  t  lie  correponding  sink  are  t he  same  exempt  t  hat  for  t  he 
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direction  of  the  motion  which  mu-t  be  reversed  Reversing  the  arrows 
in  Figure  -1.  for  example,  obtain  the  phase  portrait  of  a  system  with 
coincident  real  positive  eigenvalues.  A  source  is  obviously  an  unstable 
equilibrium. 

[\  The  eigenvalues  are  pure  imaginary.  The  origin  is  called  a  center.  All 
the  solutions  are  periodic  with  the  same  period.  A  center  is  a  stable 
equilibrium.  For  a  reason  that  will  be  made  clear  soon,  this  last  case  is 
of  lit  tie  pract  ical  interest .  since  even  a  small  pc:  ur  bat  ion  of  the  field  will 
make  the  orbits  spiral  inward  to  (or  outward  from)  the  origin,  changing 
the  qualitative  properties  of  the  solution’s  curves.  In  other  words,  a 
center  is  not  a  structurally  stable  property. 

The  crucial  point  is  that  this  classification  is  nhau.-tn  c.  Kverv  solution 
to  Kquation  (  I..T2)  (in  the  linear  looks  like  a  saddle,  a  -ink.  a  sourc  e,  or 

a  c  enter.  The  same  classification  holds  for  t  he  non-linear  <  a  so  w  it  h  respect  to 
t  he  eigenvalues  of  t  he  derivat  ive  of  t  he  right-hand  side  of  (  I  ..'1.2 1 .  e  on- 'ben  d 
as  a  linear  operator.  This  is  equivalent  to  consider  a  linear  approximation 
of  the  sy-tem  in  the  neighbors  of  t  fie  origin.  However  non-linear  w-icne- 
.ire  interesting  in  theirself.  since  they  can  show  also  a  different  qualitative 
behavior.  A  non-linear  system.  indeed,  can  have  in  addition  hunt  ri/dc  \ 
Inluilivch.  a  limit  cycle  is  a  closed  orbit  towards  which  cither  solution- 
curves  spiral  with  t  lie  same  as  \  in  plot  ir  period .  Defining  a  ^ -it  not  >e  t .  /.(/'. 
as  l^_(.r)  {a  A  sticli  that  /,.  •  sc  with  .r(/„l  •  a\  and  similarly  an 

o -limit  .0/  I.  ,(jr  I  as  /.  ,(.r )  (ft  A  such  that  !,  ■  v  with  x\t ,  )  •  it). 
a  limit  cycle'  is  a  c  losed  cubit  such  that  y  /. ^  ( .r )  or  I  ,(.r)  for  -omc 

,r  .  I  nder  somc'what  more  restrictive'  ceuuiit ion-,  a  limit  cvcle  can  be  a 
per  unite  attractor  (lor  a  rigorous  definit  ion  of  il  see  llirscfi  ami  Smale  1'lTll. 
Intuitively,  a  pe  riodic  attrac  tor  is  a  limit  cycle  -tie  h  that  nearbv  t rajee  t one- 


not  only  have  the  same  asymptotic  period  but  also  are  in  phase.  Saddles, 
sinks,  sources  and  periodic  attractors  are  very  important  for  a  qualitative 
description  of  planar  systems.  Indeed  it  can  be  shown  that  such  properties 
are  structurally  stable ,  that  is  they  persist  after  a  perturbation  of  the  right- 
hand  side  of  (4.3.2).  As  far  as  planar  systems  are  concerned  they  also  fully 
characterize  limit  sets.  By  means  of  the  Poincare- Bend ixon  theorem  it  can 
be  shown  that  compact  limit  sets  other  than  limit  cycles  are  saddles,  or  sinks, 
or  sources  or  trajectories  joining  them. 

4.4.  Equilibria  and  their  Interpretations 

In  the  definition  of  dynamical  system  the  right-hand  side  of  Equation  (4.3.2) 
can  be  interpretated  as  a  vector  field  tangent  to  the  family  of  c  urves  in  the 
plane,  solutions  to  (4.3.2)  itself.  It  is  straightforward  to  see  that  both  the 
smoothed  optical  flow  and  motion  field  (i.e.  after  the  filtering  operation)  can 
be  considered  as  istances  of  such  a  vector  field  '5.  Indeed,  it  is  sufficient  to 
insure  that  both  the  fields  are  continuous  with  continuous  first  derivatives. 
The  classification  of  the  solutions  can  now  be  interpreted  in  terms  of  char¬ 
acteristic  points  of  the  2-D  motion  field.  A  source,  for  example,  corresponds 
to  a  focus  of  expansion  of  the  field.  The  structural  stability  of  the  source, 
in  turn,  says  that  a  focus  of  expansion  persists  even  if  the  field  is  perturbed. 
From  this  perspective  a  focus  of  expansion  is  expected  to  be  detectable  in 
a  2-D  motion  field  reconstructed  with  different  algorithms  and  in  the  opti¬ 
cal  flow  as  well,  when  they  can  be  considered  as  perturbed  examples  of  the 

"We  stress  the  fart  that  the  analogy  with  the  dynamical  system  is  between  phase, 
portraits  of  dynamical  systems  and  motion  Hows.  The  parameter  t  in  the  defini¬ 
tion  of  dynamical  system  is  not  the  physical  time.  We  considered  motion  flows, 
such  as  the  2-D  motion  field  or  the  optical  flow  at  a  fixed  time,  comparing  them 
with  the  vector  field  tangent  to  the  phase  portrait  of  some  system  we  are  not 
interested  in  the  physical  meaning  of  the  underlying  dynamical  system 


■‘true”  2-D  motion  field. 


4.5.  Discussion 

If  our  point  of  view  is  correct,  the  only  critical  property  of  the  optical  flow  is 
that  it  have  the  same  qualitative  properties  of  the  2-1)  velocity  field.  Notice 
that  this  requirement  also  satisfies  two  important  uses  of  the  optical  flow: 
to  detect  disroot inmties  and  to  help  long-range,  watching  of  the  stereo  type, 
needed  for  the  computation  of  struct  ure-from-mot  ion.  Quantitative  equiva¬ 
lence.  which  is  impossible  in  general,  is  in  any  case  irrelevant  for  this  use  of 
the  optical  flow.  As  a  consequence,  many  different  “ optical  flows"  way  be 
dejined.  liquation  (2.1.6)  does  not  have  any  priviliged  role:  other  definitions 
could  he  preferred  on  the  basis  of  criteria  such  as  computability  (from  image 
data)  or  ease  of  implementation  (for  given  hardware  constraints). 

This  point  of  view  has  clear  implications  for  biological  visual  systems: 
movement  detecting  celN  (say.  in  the  retina)  do  not  have  to  compute  the 
specific  minimal  optical  How  defined  by  equation  (2.1.6):  other,  possibly 
simpler,  estimates  of  the  velocity  field  that  preserve-  its  qualitative-  proper¬ 
ties  ;,re  equally  good  candidate's  (sue  h  as  corredat ion-like'  algorithms).  This 
argument  may  explain  w  hy  the  models  proposed  to  explain  motion  dependent 
behaviour  in  inse-cts  (Hassenstein  and  Kciehardt.  1 9.76).  motion  perception 
in  humans  (Van  Santen  and  Spending.  19*1)  and  physiology  ol  cedis  (Harlow 
and  I. evic  ts.  1967:  Torre’  and  Hoggio.  197*)  are  all  implement ing  computa- 
t  ions  e|uite’  eliffeneuit  from  t  he'  minimal  opt  ical  flow  as  it  is  usually  defined  (sen1 
equation  2.1.6).  In  addition  all  these  models  do  not  typically  measure  ve- 
loe  it v  not  even  in  the  case-  o!  uniform  translation  in  a  I rontopa railed  plane. 
I. veil  for  simple  motions  of  the-  latte-t  tvpe  the  output  of  models  such  as  the 
correlation  moeieds  depends  on  both  the  ve  loc  ity  and  the-  spatial  structure  of 


the  moving  pattern.  One  is  tempted  to  consider  this  as  a  weakness  of  these 
models  compared  to  the  definition  of  minimal  optical  flow.  Equation  2.1.7. 
Our  results,  however,  show  that  this  is  not  the  case:  first,  the  minimal  optical 
flow  is  correct  only  in  a  very  special  situation;  second,  all  these  models  ma\ 
have  the  same  qualitative  properties  of  the  motion  field,  which,  from  our 
point  of  view,  is  the  only  critical  requirement  for  a  “good’'  measurement  of 
mot  ion.  The  next  question  is  of  course  whether  these  biological  models  are  in 
fact  “close”  enough  to  the  motion  field  to  share  the  same  qualitative  proper¬ 
ties.  We  do  not  know  the  answer  yet.  We  conjecture,  however,  that  they  are 
indeed  usually  similar  enough  to  preserve  the  main  qualitative  properties  of 
the  motion  field.  The  conjecture  is  based  on  results  (Poggio  and  Reichardt. 
1973  and  Poggio,  1985)  showing  that  most  of  the  biological  models  proposed 
so  far  can  be  considered  as  special  instances  or  approximations  of  a  general 
class  of  nonlinear  models  (characterized  as  Volterra  systems  of  the  second 
order);  and  that  the  minimal  optical  flow,  as  defined  in  equation,  is  also 
approximately  a  Volterra  functional  of  the  second  order  (Poggio,  1985). 

It  is  important  to  stress  that  the  approach  outlined  in  the  second  part 
of  this  paper  for  classifying  the  qualitative  properties  of  the  optical  flow  is 
only  one  of  the  possible  methods.  While  we  plan  to  develop  further  that 
particular  approach,  others  should  be  explored  as  well:  in  particular  flows 
that  do  not  correspond  to  dynamical  systems  on  the  plane  may  be  better 
suited  for  capturing  important  and  stable  properties  of  the  velocity  field 
such  as  motion  discontinuities.  In  this  case,  the  classification  of  qualitative 
properties  should  take  place  without  a  preliminary  smoothing  operation. 

In  addition  to  the  classification  of  stable  qualitative  properties  of  the 
velocity  field,  much  work  needs  to  be  done  at  the  level  of  their  interpret  at  ion 
in  terms  of  3-1)  structure  and  3-1)  velocity.  Some  of  the  qualitative  properties 
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of  the  (smoothed)  velocity  held  have  an  easy  interpretation  in  those  terms: 
an  obvious  example  is  again  a  focus  of  expansion  that  is  usually  related 
to  ‘‘crashing"  motion.  It  is  likely  that  many,  more  subtle  relations  exist 
between  the  qualitative  properties  of  the  flow  and  the  underlying  3-1)  motion 
and  structure.  For  example,  preliminary  results  In  Torre  et  al.  (personal 
communication)  suggest  that  the  number  of  focuses  in  the  (smoothed)  field 
may  be  characteristic  for  the  rigidity  of  motion  in  the  visible  scene. 

Finally,  we  should  mention  an  obvious  extension  of  the  approach  de¬ 
scribed  in  the  second  part  of  the  paper.  We  have  only  considered  so  far  the 
velocity  field  ‘‘frozen"  at  a  given  instant  of  time.  The  succession  of  image 
frames  provides  in  fact  a  time-dependent  field:  the  evolution  in  time  of  the 
qualitative  properties  we  have  described  how  they  are  created,  disappear 
and  transform  should  be  characterized  in  qualitative  terms,  for  instance 
using  the  language  of  catastrophe  and  bifurcation  theory.  The  use  of  time- 
dependent  fields  should  be  practically  much  more  robust,  because  of  the 
redundant  information  available  in  a  sequence  of  very  closely  spaced  frames 
(in  time).  Our  analysis  should  be  extended  to  qualitative  properties  that  are 
structurally  stable  not  only  at  a  given  time  but  also  in  tlx'  time  dependent 
field. 
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4.6.  Appendix  Al:  Perspective  and  Orthographic  Projections 


In  this  section  we  explain  in  more  detail  the  geometry  of  perspective  pro¬ 
jection  used  in  the  paper.  Let  n  be  the  unit  normal  to  the  projection  plane 
and  /  the  focal  length.  In  order  to  obtain  ihe  orthographic  projection  as 
the  limit  of  the  perspective  one  for  /  •  oc.  the  focus  cannot  be  located  at 

the  origin  of  the  system  of  coordinates.  To  simplify  the  geometry  without 
losing  in  generality,  let  the  origin  lit'  on  the  projection  plane.  The  vector 
pointing  from  the  focus  to  a  point  x  (r.y.z)  is  now  fn  4  x.  To  obtain  the 
expression  of  the  projected  point  xJt  notice  that  from  Figure  1  is  easy  to  see 


that 


From  that,  we  have 


and  finally 


or 
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The  orthographic  projection  equation  can  be  easily  obtained  for  /  •  oc.  i.r. 

x,,rt  lim  x,,  f  (n  •  (x  •  n) )  (n  ■  (x  ■  n) ) . 

/  -  x  /  •  x  •  n 

Combining  the  last  two  equations,  we  obtain  the  general  relationship 
between  perspective  and  orthographic  projection,  that  is 

/ 


i 


SI 
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